In 1964, Motown's main songwriting/production team of Holland–Dozier–Holland created a complete instrumental track without any idea of what to do with it. They decided to craft the song as a more mainstream pop song for the Four Tops and proceeded to create "Baby I Need Your Loving" from the lyric-less instrumental track.

On its mid-1964 release, "Baby I Need Your Loving" made it to #11 on the Billboard pop charts. However, the song proved to be much more popular on trend-setting radio stations in key U.S. markets and has since grown in popularity over the years to be one of the group's classic tracks. After the single's success, the Tops were pulled away from their jazz material and began recording more material in the vein of "Baby I Need Your Loving". The first follow-up single, "Without the One You Love (Life's Not Worth While)", just missed both the pop and R&B Top 40 charts, but "Ask the Lonely", written and produced by Motown A&R....